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SPATIAL MOTION RECOGNITION SYSTEM AND METHOD 
USING A VIRTUAL HANDWRITING PLANE 

CROSS-REFERENCE TO RELATED APPLICATIONS 
This application claims the priority of Korean Patent Application No. 2003-16410, 
filed on March 17, 2003, in the Korean Intellectual Property Office, the disclosure of which is 
incorporated herein in its entirety by reference. 

BACKGROUND OF THE INVENTION 
L Field of the Invention 

An apparatus consistent with the present invention relates to a spatial motion 
recognition system and, more particularly, to a system and method for recognizing motions 
occurring in space. 

2. Description of the Related Art 

Recently, the market for personal mobile devices such as personal digital assistants 
(PDAs), cellular phones, computer notebooks, and so on, has been widely spread. Such 
personal mobile devices are best suited for ubiquitous environments for utilizing information 
anytime and anywhere since they are easy to carry. That is, recent mobile devices enable 
users to utilize information even during their movements so that the users can use information 
any time and anywhere, unlike the past environments which confined information utilization 
to desktop personal computers (PCs) installed at a fixed location. 

However, such mobile devices have become smaller in an overall size in order to be 
easily carried, which causes a problem that users feel somewhat awkward or inconvenienced 



in recognitions and command inputs through a small display unit. In such mobile devices, 
portability runs counter to being easy to input and output information, so research and 
development has been continuously made on methods for overcoming these problems. 

[05] Recently, a pen-shaped input system has emerged as a means through which users use 

the mobile devices more conveniently. A general pen-shaped input system is designed to 
recognize writing motions or selecting motions which are inputted through a tool on the two- 
dimensional plane or pad so that the pen-shaped input system displays writings on a display 
unit or performs corresponding operations based on recognized information. 

[06] Fig. 1 illustrates an input style for a conventional pen-shaped input system. The pen- 

shaped input system has a stylus pen part 10 capable of being held in a user's hand so that a 
user can write or select motions with it, and a display part 22, which may be a PDA 20, for 
example, that recognizes menu selection commands and/or writing motions from a contact 
state of a portion of the stylus pen part 10 on the two-dimensional plane of the display part 22 
and displays corresponding operations based on a selected menu or handwritings. The pen 
part 10 operates as a pointing tool or handwriting tool without an extra power supply, and the 
PDA 20 processes a signal corresponding to a portion of the display part 22 on which the tip 
portion of the pen part 10 is placed. 

[07] The pen-shaped input system described above decides what information is inputted 

depending upon positions in which the pen part 10 contacts with the two-dimensional plane 
of the display part 22. That is, if a current view displayed on the display part 22 is a view 
displaying menus, the PDA 20 decides information corresponding to a position with which 
the tip portion of the pen part 10 is in contact as menu selection information, and performs a 
corresponding command based on information on the selected position. Further, if a view 
capable of recognizing handwritings is displayed, the PDA 20 obtains information on contact 
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positions of the pen part 10 that are continuously changing, and displays on the display part 
22 the handwritings from the obtained information. 
[08] However, the above pen-shaped input system is inconvenient since users have to 

perform their menu selections or handwriting motions on the limited view of the display part 
22. 

[09] Meanwhile, other systems have been proposed for measuring or detecting the 

coordinate changes of the pen tip in three axial angles outside the pen or through ultrasonic 
waves or the like, for other types of conventional pen-shaped input systems. Further, there 
has been proposed a system for detecting the changes of the pen tip used in charge-coupled 
device (CCD) cameras. For yet other types of conventional pen-shaped input systems, 
United States Patent Ser. Nos. 6,181,329 (laid-open January 30, 2001) and 6,212,296 (laid- 
open April 3, 2001) disclose systems extracting motion information and recognizing input 
information inside the pen based on motions performed on the two-dimensional plane or in 
three-dimensional space. 

[10] However, problems exist in the systems extracting coordinate changes outside the pen, 

such as the systems using three axial angles, ultrasonic waves, CCD cameras, and so on, 
since there is a high possibility of recognition errors if the pen is interfered with by a hand or 
other object. Further, the pen-shaped input systems disclosed in USP Nos. 6,181,329 and 
6,212,296 extract motion information and recognize motions based on pen motions 
performed on the two dimensional plane or in the three dimensional space, so users can input 
information without any limit to handwriting space, but, when pen motions occur in the three- 
dimensional space, there exists a problem that handwritings can be wrongly recovered and 
displayed differently from users' intents since users can not make handwriting motions 
precisely on an arbitrary plane in space. For example, as a user performs motions for writing 
the number "2" as shown in Fig. 2 in space, the positions of points appearing when 



segmenting the handwriting motions of the number "2" in predetermined time intervals do 
not exist on any one plane corresponding to each other, so they may appear somewhat 
compact or as a completely different symbol depending upon recognition directions, which 
also causes a high possibility of recognition error occurrences. Fig. 3 illustrates a shape of 
the number "2" based on the handwriting motions of Fig. 1 which appears when viewed in a 
different direction. Accordingly, the pen-shaped input systems disclosed in the above- 
mentioned United States Patents can reduce recognition errors when the pen tip contacts the 
two-dimensional writing surface, but they are not easy to utilize in space. 

SUMMARY OF THE INVENTION 

[11] Illustrative, non-limiting embodiments of the present invention overcome the above 

disadvantages and other disadvantages not described above. Also, the present invention is 
not required to overcome the disadvantages described above, and an illustrative, non-limiting 
embodiment of the present invention may not overcome any of the problems described above. 

[12] An apparatus consistent with the present invention provides a spatial motion 

recognition system and a method capable of improving recognition capability of handwriting 
motions in space without any extra writing surface provided. 

[13] In order to achieve the above aspect, an exemplary embodiment of a spatial motion 

recognition system according to the present invention comprises a motion detection unit for 
outputting, as an electric signal position, changes based on motions of a body of the system in 
space.A control unit for tracking three-dimensional motions of the system body based on the 
electric signal outputted from the motion detection unit, produces a virtual handwriting plane 
having the shortest distances with respect to respective positions in predetermined time 
intervals based on three-dimensional track information obtained through tracking, and 
projects the respective positions in the predetermined time intervals onto the virtual 
handwriting plane to recover the motions in space. 



[14] Further, the control unit rotation-converts the tracks of the virtual handwriting plane 

into a two-dimensional plane of x and y axes in order to reproduce the tracks projected onto 
the virtual handwriting plane on the two-dimensional plane. 

[15] In the meantime, in order to achieve the above aspect, a spatial motion recognition 

method for a motion recognition system comprises steps of obtaining three-dimensional track 
information on a system body in space; producing a virtual handwriting plane having the 
shortest distances with respect to respective positions in predetermined time intervals based 
on the obtained three-dimensional track information; and projecting the positions in the 
predetermined time intervals onto the virtual handwriting plane and recovering the motions in 
space. 

[16] Further, the spatial motion recognition method further comprises a step of rotation- 

converting the tracks of the virtual handwriting plane into a two-dimensional plane of x and y 
axes in order to reproduce the tracks projected onto the virtual handwriting plane on the two- 
dimensional plane. 

[17] As provided above, the spatial motion recognition system and method according to 

the present invention enables users to input information regardless of a handwriting area, 
enabling precise motion recognitions for handwriting motions in space without providing an 
extra handwriting surface, since the system and method can recover users' handwriting 
motions in space just like the handwritings users perform on the surface of a plane. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[18] The features and advantages of the present invention will become more apparent by 

describing in detail exemplary embodiments thereof with reference to the accompanying 
drawings in which like reference numerals refer to like elements, and wherein: 
[19] Fig. 1 is a view for showing an exemplary input style for a conventional pen-shaped 

input system; 



[20] Fig. 2 is a view for showing handwriting motions of the number "2" in space; 

[21] Fig. 3 is a view for showing the handwriting motions of Fig. 2 when viewed in a 

different direction; 

[22] Fig. 4 is a block diagram for showing a spatial motion recognition system according 

to an embodiment of the present invention; 
[23] Fig. 5 is a view for explaining a process for finding out straight lines having shortest 

distances with respect to respective points on the two-dimensional plane; 
[24] Fig. 6 and Fig. 7 are views for showing a process for finding out a virtual handwriting 

plane and a process for projecting three-dimensional motions in the spatial motion 

recognition system of Fig. 4; and 
[25] Fig. 8 is a view for showing vector relations for obtaining projection positions in 

order to project the three-dimensional motions on the virtual handwriting plane. 

DETAILED DESCRIPTION OF ILLUSTRATIVE, NON-LIMITING EMBODIMENTS OF 

THE INVENTION 

[26] Hereinafter, the present invention will be described in detail with reference to the 

accompanying drawings. 

[27] Fig. 4 is a schematic block diagram for showing a spatial motion recognition system 

according to an embodiment of the present invention. A system 100 includes a motion 
detection unit 110, a display unit 120, a transmission/reception unit 130, and a control unit 
140. 

[28] The motion detection unit 1 10 continuously detects the position changes of a system 

body. In order to do so, the motion detection unit 110 has plural gyro sensors, plural 
acceleration sensors, and a computing circuit for performing computations on electric signals 
transferred from respective sensors. 
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[29] The display unit 120 displays the motions of the system body on a screen. Here, the 

display unit 120 may be built in a recognition system to be incorporated into one body, and, 
further, include a displayable means built in a different system. 

[30] The transmission/reception unit 130 transmits to an external system motion 

information recognized under control of a control unit 140 or a control signal corresponding 
to the motion information. 

[31] The control unit 140 recognizes system body motions occurring in space based on 

information of system body position changes that is detected from the motion detection unit 
110, and then displays the recognized motion information on the display unit 120 or transmits 
to a different system a control signal corresponding to the recognized motion information or 
the motion information through the transmission/reception unit 130. At this time, the control 
unit 140 performs a process for producing a virtual handwriting plane from the position 
change information detected through the motion detection unit 1 10 in order to recognize the 
system body motions, and a process for projecting the motion track information of the system 
body onto the produced virtual handwriting plane. Further, the control unit 140 performs a 
process for carrying out a rotation conversion of the motion tracks projected on the virtual 
handwriting plane into a two-dimensional plane in order to display the obtained motion 
information on the display unit 120. 

[32] The process for determining the virtual handwriting plane in the control unit 140 can 

be referred to as a process for determining a plane most adjacent to respective points, in a 
state that handwriting tracks in the three-dimensional space are first established as a set of 
points. This can be achieved by expanding into the three-dimensions a process for finding 
out straight lines having the shortest distances with respect to the respective points of tracks 
on the two-dimensional plane. 



7 



[33] First, when points obtained through tracks handwritten on the two-dimensional plane 

are referred to as Pi = (xi, yO wherein i = 1, m, a straight line having the shortest distance 
from Pi can be expressed in a straight line of the form y = ax + b. Further, the straight line of 
y = ax + b can be obtained by using Equation 1 and Equation 2 as follows from coordinate 
values of points obtained based on handwriting motions on the two-dimensional plane. 

[Equation 1] 

m 

E = X(ax i+ b- yi f 

1=1 

5 

[34] wherein E denotes a sum of the square of distances of the points spaced away from a 

straight line of y = ax + b, a denotes a slope of the line, and b denotes an intercept of the line. 

[Equation 2] 

M = 0 M = 0 

da and db 

[35] That is, the straight line of the form y = ax + b can be obtained through the partial 

differentiation of E by a and b after calculating E which is a sum of the square of distances to 
a virtual line from respective points. 

[36] However, when performing handwriting motions in the three-dimensional space, a 

user performs the handwriting motions while assuming that there is a virtual plane in the 
three-dimensional space, but the points actually appearing when handwriting tracks are 
segmented in predetermined time intervals do not correspond to the virtual plane the user has 
assumed. Fig. 6 is a view for representing tracks based on a user's handwriting motions in 
the three-dimensional space by use of points with reference to a virtual handwriting plane. 
Accordingly, the present invention produces a virtual handwriting plane to recognize users' 
handwriting motions in the three-dimensional space as the handwriting motions in the two- 
dimensional plane. A process for determining a virtual handwriting plane can be obtained by 

expanding into the three-dimension a process for finding straight lines having the shortest 
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distance in use of the linear regression method with respect to points on the two-dimensional 
plane. 



motions in the three-dimensional space, the virtual handwriting plane can be assumed in 
Equation 3 as follows when points obtained based on motions handwritten in the three- 
dimensional space are referred to as Pi = (x i5 y i9 z,) wherein i = 1,..., m. 

[Equation 3] 

z = ax + /3y + y 

[38] Further, obtained in the process is a sum of the square of distances to the virtual plane 

from the m points in the three-dimensional space. Equation 4 as follows expresses the sum of 
the square of distances to the virtual plane from the m points in the three-dimensional space. 

[Equation 4] 



[37] 



In the process for producing a virtual handwriting plane based on handwriting 



m 



i=i 



[39] 



Further, in order for the E to have the minimum value with respect to the parameters a, 



p, and y, Equation 5 as follows has to be satisfied. 



[Equation 5] 




dE_ 




dy 



[40] 



Further, the parameters a, p, and y for a virtual handwriting plane from the Equations 



3, 4, and 5 can be obtained through Equation 6 as follows. 



[Equation 6] 
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Z*<- 2 Z*<* z*. 

1=1 1=1 1=1 

w /w m 

Z*^- Z^ 2 Z^ 

1=1 i=i 

2>. 5> f 



1=1 
m 
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1=1 

r=l 



[41] wherein x t , yi, zj are coordinates of the system body that is tracked at a predetermined 

time in the three-dimensional space, and a, p, and y are parameters for a virtual handwriting 
plane. 

[42] If the parameters for a virtual handwriting plane are obtained, the control unit 140 

recognizes a virtual plane having the shortest distances with respect to other points of three- 
dimensional tracks. Thereafter, the control unit 140 performs a process for projecting the 
respective points of the three-dimensional tracks onto the points nearest a virtual handwriting 
plane as shown in Fig. 7. The nearest distance to the virtual handwriting plane from the 
points of the three-dimensional tracks becomes scalar times the normal vector with respect to 
the virtual handwriting plane at respective points as shown in Fig. 8, so Equation 7 as follows 
can be expressed: 

[Equation 7] 

P-P'=kN, 

[43] wherein P is a vector based on tracks on the three-dimensional space, P' is a vector 

projected on a virtual handwriting plane, and N is a normal vector for the virtual handwriting 
plane. Points, P'i(xi\ y { \ Zi') wherein i = 1, m, which are projected on the virtual 
handwriting plane by the above Equation 7 can be obtained through Equation 8 as follows: 

[Equation 8] 
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.1 I 



[44] 
[45] 



[46] 



[47] 



[48] 



[49] 



x, = x, - 



y, = yi 



Z, = Z: 



a(ax i + by i + cz t + d) 
a 2 +b 2 +c 2 

b{ax i + bx i + cZj + d) 
a 2 +b 2 +c 2 

c(ax t + bx i + CZj + d) 
a 2 +b 2 +c 2 



The projection process as above recovers tracks in space into tracks on plane. 

Thereafter, the control unit 140 performs a process of rotating the tracks on a virtual 
handwriting plane by 9 degrees about the y axis and by <1> degrees about the x axis so that the 
tracks on the virtual handwriting plane can be reproduced on the two-dimensional plane. 

The rotation-converted tracks are obtained through Equation 9 as follows: 



[Equation 9] 



X," 




I 


0 


0 


yi 




0 


COS0 


- sin (j. 


_z," 




0 


sin 0 


COS0 



cos 6 0 sin 0 

0 1 0 
- sin 0 0 cos 6 



(f> = arctan 2{-b-c) 



6 = arctan 2(a, yjb 2 +c 2 ) 

wherein (xj\ yi', Zj') is a three-dimensional coordinate when tracks are segmented in 
predetermined time intervals and then the i* position of (xi, yi, zj) is projected on a virtual 
handwriting plane, (xi", yi", Zi") is a coordinate of a point obtained when the i* position of the 
projected tracks is rotated by # degrees about the y axis and rotated by </> degrees about the x 



axis. 



(0). 



Since the rotation-converted point exists on an x-y plane, a value of Z[ becomes zero 



As above, if track information of a virtual handwriting plane is rotated and moved to 
the two-dimensional plane of x and y axes, the control unit 140 can display three-dimensional 
motions as two-dimensional information on a screen of the display unit 120 through the 
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rotated and moved information, and the transmission/reception unit 130 transfers the two- 
dimensional information to a different system. 
[50] As described above, since the spatial motion recognition system and method 

according to the present invention recognizes motions in space as handwriting motions on a 
two-dimensional plane, the system and method are capable of precise motion recognition 
without requiring an extra handwriting surface, to thereby promote users' convenience, and 
the system and method enable a user to input information regardless of a handwriting area so 
as to greatly improve problems associated with inputting information in ubiquitous 
environments. 

[51] Further, the system and method recognize handwriting motions in space as those on 

the existing two-dimensional plane, so that they can solve the obscure handwriting 
recognition problem. 

[52] While the invention has been shown and described with reference to exemplary 

embodiments thereof for illustrative purposes, it will be understood by those skilled in the art 
that various changes in form and details may be made therein without departing from the 
spirit and scope of the invention as defined by the appended claims. 
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